feat: Stable system prompt and tool ordering for more cache hits by jsibbison-square · Pull Request #5192 · block/goose

jsibbison-square · 2025-10-15T22:45:56Z

Summary

When running multiple goose instances for the same tasks (eg. recipes) we currently do not get any llm cache hits on the initial request. From my testing this happens for multiple reasons:

We have a second-based timestamp in the system prompt
Tools are added in a random order
Extension instructions come in a random order

This PR makes the timestamp truncated at the hour to allow for decent time accuracy but greatly improved cache hit potential and sorts the extensions (for instructions) and tools by name so they always appear in the same order.

Type of Change

Testing

I've used an api key with openai and downloaded their usage csv to verify that caching on the same initial request run in subsequent sessions with the developer mcp results in almost 100% cached input token usage.

Updated comment to clarify prompt cache behavior.

DOsinga

I think @tlongwell-block is also looking at some of this

DOsinga · 2025-10-16T12:02:21Z

crates/goose/src/agents/prompt_manager.rs

    }
+
+    #[test]
+    fn test_build_system_prompt_orders_extensions_by_name() {


delete this test

I thought it was useful to avoid regressing but have deleted it.

tlongwell-block · 2025-10-16T13:29:12Z

Yeah, #5027 removes the timestamp entirely from the system prompt

jsibbison-square · 2025-10-16T21:17:21Z

@tlongwell-block So should I just delete the timestamp line in this PR so we get caching sooner or do you want me to not make that change and have your PR do that? I matched your change to delete the timestamp for system_prompt.md so shouldn't conflict when it merges

Once you respond I'll be able to merge.

…com:block/goose into jsibbison-20251015-no-dynamic-system-prompt * 'jsibbison-20251015-no-dynamic-system-prompt' of github.com:block/goose: Clarify comment on prompt cache filtering

tlongwell-block · 2025-10-17T15:39:16Z

@tlongwell-block So should I just delete the timestamp line in this PR so we get caching sooner or do you want me to not make that change and have your PR do that? I matched your change to delete the timestamp for system_prompt.md so shouldn't conflict when it merges

Once you respond I'll be able to merge.

Until #5027 is merged, this would mean goose won't know the time at all

@DOsinga, what do you think about merging #5027 right after this one? If we're still not sure about MOIM as a concept, I'd say leave the timestamp in here @jsibbison-square. I will say that the MOIM build is my daily driver and works very well, including on the benchmarks.

jsibbison-square · 2025-10-19T09:24:12Z

Ok I'll put the timestamp back at just the hourly granuality that way we get greater caching and you don't need to rush the other PR.

This reverts commit 3ac97ce.

This reverts commit c132834.

DOsinga · 2025-10-20T17:36:21Z

crates/goose/src/prompts/system.md

 You are a general-purpose AI agent called goose, created by Block, the parent company of Square, CashApp, and Tidal.
 goose is being developed as an open-source software project.

+The current date is {{current_date_time}}.


we could consider adding this to the end so we get most of the system prompt cached

Makes sense, will leave it unchanged here so tylers pr doesn't get a conflict when he removes the timestamp (we changed the timestamp to be at an hour granuality so we still get improved caching benefits).

- Includes stable system prompt and tool ordering (#5192) - Memory extension prompt updates (#4945) - Smart task organizer recipe (#5032) - Recipe open command docs (#5264) - Various provider and UI updates

* main: Fix extension manager resource deadlock (#5066) add new prompt for memory extension (#4945) feat: Stable system prompt and tool ordering for more cache hits (#5192) Remove gtag analytics error (#5268) Feat/smart task organizer recipe (#5032) docs: `goose recipe open` command (#5264) docs: provide more clarity in tagging step of releases (#5269) docs: add GOOSE_DEBUG environment variable (#5265)

* main: (40 commits) Fix extension manager resource deadlock (#5066) add new prompt for memory extension (#4945) feat: Stable system prompt and tool ordering for more cache hits (#5192) Remove gtag analytics error (#5268) Feat/smart task organizer recipe (#5032) docs: `goose recipe open` command (#5264) docs: provide more clarity in tagging step of releases (#5269) docs: add GOOSE_DEBUG environment variable (#5265) Lifei/UI parameter input (#5222) turn off WAL (#5203) Skip subagents for gemini (#5257) Revert "Standardize Session Name Attribute" (#5250) improve provider request logging a bit (#5236) Fix OpenAI empty choices panic (#5248) Revert "Rewrite extension management tools" (#5243) Standardize Session Name Attribute (#5085) Docs: Include step-by-step model configuration instructions for first… (#5239) Delete message visibility filter (#5216) delete flaky pricing integration tests (#5207) chore: upgrade most npm packages to latest (#5185) ...

* main: live testing script (#5263) Fix extension manager resource deadlock (#5066) add new prompt for memory extension (#4945) feat: Stable system prompt and tool ordering for more cache hits (#5192) Remove gtag analytics error (#5268) Feat/smart task organizer recipe (#5032) docs: `goose recipe open` command (#5264) docs: provide more clarity in tagging step of releases (#5269) docs: add GOOSE_DEBUG environment variable (#5265)

* 'main' of github.com:block/goose: fix: include Windows GNU CLI artifact in releases (#5276) Fix bedrock tool call error response (#5067) fix: do not load active extensions when no extensions in the recipe (#5282) Fix pagination - fetch all open issues, not just first 100 Fix label case sensitivity - use lowercase labels Add GitHub Health Dashboard workflow (#5286) live testing script (#5263) escaping markdown fixing syntax Add GitHub Health Dashboard workflow Fix extension manager resource deadlock (#5066) add new prompt for memory extension (#4945) feat: Stable system prompt and tool ordering for more cache hits (#5192) Remove gtag analytics error (#5268)

…ck#5192) Signed-off-by: Blair Allan <Blairallan@icloud.com>

jsibbison-square added 3 commits October 16, 2025 09:40

feat: Stable system prompt and tool ordering for more cache hits

04739b7

Order extension instructions

9b13c99

Format

167d01c

jsibbison-square marked this pull request as ready for review October 15, 2025 23:56

cloud-on-prem approved these changes Oct 16, 2025

View reviewed changes

Clarify comment on prompt cache filtering

8790465

Updated comment to clarify prompt cache behavior.

DOsinga reviewed Oct 16, 2025

View reviewed changes

jsibbison-square added 4 commits October 17, 2025 08:26

Remove test

26f5e47

Merge branch 'jsibbison-20251015-no-dynamic-system-prompt' of github.…

7bdc3ac

…com:block/goose into jsibbison-20251015-no-dynamic-system-prompt * 'jsibbison-20251015-no-dynamic-system-prompt' of github.com:block/goose: Clarify comment on prompt cache filtering

Remove date time in system prompt

c132834

Revert timestamp format change

3ac97ce

jsibbison-square added 2 commits October 19, 2025 20:25

Revert "Revert timestamp format change"

61828d0

This reverts commit 3ac97ce.

Revert "Remove date time in system prompt"

43d9556

This reverts commit c132834.

DOsinga approved these changes Oct 20, 2025

View reviewed changes

jsibbison-square merged commit e9ca2bc into main Oct 20, 2025
11 checks passed

jsibbison-square deleted the jsibbison-20251015-no-dynamic-system-prompt branch October 20, 2025 21:48

BlairAllan pushed a commit to BlairAllan/goose that referenced this pull request Oct 25, 2025

feat: Stable system prompt and tool ordering for more cache hits (blo…

1d12c35

…ck#5192) Signed-off-by: Blair Allan <Blairallan@icloud.com>

BlairAllan pushed a commit to BlairAllan/goose that referenced this pull request Nov 29, 2025

feat: Stable system prompt and tool ordering for more cache hits (blo…

418c034

…ck#5192) Signed-off-by: Blair Allan <Blairallan@icloud.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Stable system prompt and tool ordering for more cache hits#5192

feat: Stable system prompt and tool ordering for more cache hits#5192
jsibbison-square merged 10 commits intomainfrom
jsibbison-20251015-no-dynamic-system-prompt

jsibbison-square commented Oct 15, 2025 •

edited by jamadeo

Loading

Uh oh!

DOsinga left a comment

Uh oh!

DOsinga Oct 16, 2025

Uh oh!

jsibbison-square Oct 16, 2025

Uh oh!

tlongwell-block commented Oct 16, 2025

Uh oh!

jsibbison-square commented Oct 16, 2025 •

edited

Loading

Uh oh!

tlongwell-block commented Oct 17, 2025

Uh oh!

jsibbison-square commented Oct 19, 2025

Uh oh!

DOsinga Oct 20, 2025

Uh oh!

jsibbison-square Oct 20, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

jsibbison-square commented Oct 15, 2025 • edited by jamadeo Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Type of Change

Testing

Uh oh!

DOsinga left a comment

Choose a reason for hiding this comment

Uh oh!

DOsinga Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

jsibbison-square Oct 16, 2025

Choose a reason for hiding this comment

Uh oh!

tlongwell-block commented Oct 16, 2025

Uh oh!

jsibbison-square commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tlongwell-block commented Oct 17, 2025

Uh oh!

jsibbison-square commented Oct 19, 2025

Uh oh!

DOsinga Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

jsibbison-square Oct 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

jsibbison-square commented Oct 15, 2025 •

edited by jamadeo

Loading

jsibbison-square commented Oct 16, 2025 •

edited

Loading